Update lora def by lucylq · Pull Request #18247 · pytorch/executorch

lucylq · 2026-03-17T20:10:32Z

Summary

Update lora def to use nn.linear instead of torch.nn.functional.linear

Remove debug string from FQN. Otherwise we get different quant indices for lora and non-lora, after this change, and the foundation weights are no longer shareable...
Having nn.Linear as a submodule allows torchao quant to capture it, and we don't need custom logic filtering LoraLinear. lora_a and lora_b are also captured, which changes our results slightly. Note that only lora_a is quantized (shape [16, dim], where dim %32 ==0), as lora_b has shape [dim, 16], where 16 %32 != 0 and fails the group size check.
Having nn.Linear as submodule also means we need to remap lora weights, as weight names have an extra 'linear' in them.
Add @Property for weight, bias so that it remains BC and can be treated as a regular linear module. This is used in [load_weights_from_attention.py](

executorch/examples/models/llama/static_attention.py

Line 1154 in eb92cec

def load_weights_from_attention_mha(

Test plan

CI

pytorch-bot · 2026-03-17T20:10:38Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18247

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Awaiting Approval, 1 New Failure, 3 Cancelled Jobs, 3 Unrelated Failures

As of commit 2f28c1c with merge base eb92cec ():

AWAITING APPROVAL - The following workflow needs approval before CI can run:

Claude Code (gh)

NEW FAILURE - The following job has failed:

pull / unittest / macos / macos-job (gh)
export/tests/test_target_recipes.py::TestTargetRecipes::test_mv3_model

CANCELLED JOBS - The following jobs were cancelled. Please retry:

pull / test-samsung-models-linux / linux-job (gh)
##[error]The operation was canceled.
trunk / test-coreml-delegate / macos-job (gh)
##[error]The operation was canceled.
trunk / test-models-macos-coreml (edsr) / macos-job (gh)
##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.
pull / unittest-editable / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.
trunk / unittest-release / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-03-17T20:11:56Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Pull request overview

Updates the LLaMA example LoRA linear implementation to call an nn.Linear module in forward() (instead of torch.nn.functional.linear), presumably to align with module-based patterns.

Changes:

Replace torch.nn.functional.linear(x, self.weight, self.bias) with self.linear(x).
Introduce self.linear = nn.Linear(...) during LoRALinear initialization.

Comments suppressed due to low confidence (1)

examples/models/llama/lora.py:36

linear and weight are now undefined after switching to self.linear; bias = linear.bias ... and register_parameter("weight", ...) will raise at init time. Either derive bias/weight from self.linear (or remove the extra register_parameter calls entirely) so construction works.

        self.linear = nn.Linear(in_dim, out_dim, bias=use_bias)
        bias = linear.bias if self.use_bias else None
        self.register_parameter("weight", nn.Parameter(weight))
        self.register_parameter(
            "bias", nn.Parameter(bias) if bias is not None else None
        )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

examples/models/llama/lora.py

Copilot

Pull request overview

Updates the Llama LoRA implementation to call an nn.Linear submodule directly (instead of torch.nn.functional.linear) and simplifies quantization filtering accordingly.

Changes:

Refactors LoRALinear to wrap a real nn.Linear module and use it in forward.
Adds partial state-dict backward compatibility by remapping the legacy weight key.
Simplifies the 8da*w quantization filter_fn to only consider nn.Linear modules.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
examples/models/llama/source_transformation/quantize.py	Simplifies the 8da4w/8da8w quantization filter to target `nn.Linear` modules only.
examples/models/llama/lora.py	Refactors `LoRALinear` to delegate to an internal `nn.Linear` and updates forward/state-dict behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

examples/models/llama/lora.py

Copilot

Pull request overview

Updates the LLaMA LoRA implementation to wrap a real nn.Linear submodule (instead of calling torch.nn.functional.linear directly), enabling TorchAO quantization tooling to recognize and quantize the base linear layer consistently across LoRA and non-LoRA models.

Changes:

Refactors LoRALinear to contain self.linear: nn.Linear, adds weight/bias properties for backward-compatible access, and remaps old checkpoint keys on load.
Simplifies TorchAO 8da*xw quantization filtering to target nn.Linear modules only (no special-casing LoRA modules).
Makes XNNPACK constant serialization keys content-hash-based (removes tensor-name prefix) and updates the LoRA CI expectation string accordingly.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
examples/models/llama/source_transformation/quantize.py	Simplifies TorchAO quantization filter to match `nn.Linear` modules and apply group-size compatibility logic.
examples/models/llama/lora.py	Refactors `LoRALinear` to wrap an `nn.Linear` submodule; adds BC `weight`/`bias` accessors and state-dict key remapping.
backends/xnnpack/operators/node_visitor.py	Changes named constant key generation to be solely SHA256-based to stabilize dedup/indexing behavior.
.ci/scripts/test_lora.sh	Updates expected output prefix text for the quantized LoRA test case.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

examples/models/llama/lora.py

Copilot

Pull request overview

This PR updates the LLaMA LoRA implementation to use nn.Linear submodules (instead of torch.nn.functional.linear) so TorchAO quantization can capture base + LoRA linears directly, while also adjusting serialization keying to support weight sharing and updating a LoRA CI expected output string.

Changes:

Simplify TorchAO quantization filtering to target nn.Linear modules only (relying on LoRA’s internal nn.Linear submodules).
Refactor LoRALinear to contain a self.linear submodule, add weight/bias properties for compatibility, and remap legacy checkpoint keys during loading.
Change XNNPACK constant-data named_key to remove tensor-name components; update LoRA CI expected output.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
examples/models/llama/source_transformation/quantize.py	Simplifies TorchAO quantization module filter to match only `nn.Linear`.
examples/models/llama/lora.py	Refactors LoRA linear to wrap an `nn.Linear` submodule; adds BC accessors and state-dict key remapping.
backends/xnnpack/operators/node_visitor.py	Changes constant-data `named_key` generation to be content-hash only.
.ci/scripts/test_lora.sh	Updates expected quantized-LoRA output prefix string.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

backends/xnnpack/operators/node_visitor.py

            f"Serializing constant data node {tensor} but tensor value has no bytes",
        )
        sha256_hash = hashlib.sha256(bytes(array))
-        named_key = tensor.name + "_" + sha256_hash.hexdigest()
+        named_key = sha256_hash.hexdigest()



meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 17, 2026

lucylq marked this pull request as ready for review March 17, 2026 20:10

Copilot AI review requested due to automatic review settings March 17, 2026 20:10

Copilot started reviewing on behalf of lucylq March 17, 2026 20:11 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

examples/models/llama/lora.py Outdated Show resolved Hide resolved

lucylq force-pushed the lfq.lora-def branch from 05f81b8 to d23c0d2 Compare March 17, 2026 20:31

Copilot AI review requested due to automatic review settings March 17, 2026 20:59

lucylq force-pushed the lfq.lora-def branch from d23c0d2 to bc4b3df Compare March 17, 2026 20:59

Copilot started reviewing on behalf of lucylq March 17, 2026 21:00 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

examples/models/llama/lora.py Show resolved Hide resolved

examples/models/llama/lora.py Outdated Show resolved Hide resolved

lucylq force-pushed the lfq.lora-def branch from bc4b3df to 72a9449 Compare March 17, 2026 23:22

lucylq requested a review from digantdesai as a code owner March 17, 2026 23:22

Copilot AI review requested due to automatic review settings March 18, 2026 18:13

lucylq force-pushed the lfq.lora-def branch 2 times, most recently from 45fd1f3 to 1ebb7d7 Compare March 18, 2026 18:16

lucylq requested a review from JacobSzwejbka March 18, 2026 18:16

Copilot started reviewing on behalf of lucylq March 18, 2026 18:18 View session

Copilot AI reviewed Mar 18, 2026

View reviewed changes

examples/models/llama/lora.py Show resolved Hide resolved

lucylq force-pushed the lfq.lora-def branch from 1ebb7d7 to 2e04f71 Compare March 18, 2026 22:29

Update lora def

2f28c1c

lucylq force-pushed the lfq.lora-def branch from 2e04f71 to 2f28c1c Compare March 18, 2026 22:54

Copilot AI review requested due to automatic review settings March 18, 2026 22:54

Copilot AI reviewed Mar 18, 2026

View reviewed changes

backends/xnnpack/operators/node_visitor.py

Comment on lines 625 to 629

f"Serializing constant data node {tensor} but tensor value has no bytes",

)

sha256_hash = hashlib.sha256(bytes(array))

named_key = tensor.name + "_" + sha256_hash.hexdigest()

named_key = sha256_hash.hexdigest()

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update lora def#18247

Update lora def#18247
lucylq wants to merge 1 commit intomainfrom
lfq.lora-def

lucylq commented Mar 17, 2026 •

edited

Loading

Uh oh!

pytorch-bot bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lucylq commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pytorch-bot bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18247

❌ 1 Awaiting Approval, 1 New Failure, 3 Cancelled Jobs, 3 Unrelated Failures

Uh oh!

github-actions bot commented Mar 17, 2026

This PR needs a release notes: label

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lucylq commented Mar 17, 2026 •

edited

Loading

pytorch-bot bot commented Mar 17, 2026 •

edited

Loading

This PR needs a `release notes:` label